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Perfonnance Based Methods have been around for a long time. It could be argued that ALL 
methods were performance based before the lawyers, regulators and managers got their 
bureaucratic little hands on them I If these methods have been around for so long why is 
there a debate? Well, I think that the debate is pretty well over and what we are in fact 
involved with is the problem of how PBM's should be used. There is a lot of fogginess 
shrouding this subject. I hope to shine some lig^t on the subject by posing, discussing and 
perhaps, with your help, even answering some of these questions. 

PBM's... 

What are they? What do they look Uke? What are the benefits? 
What are the liabilities? What are some issues and approaches? 
Who might use them? Where Do We Gro From Here? 



What Are Performance-Based Methods? 

Well, the quick answer is; Fm not sure! Different people, from different groups seem to 
have different opinions. In most cases, close inspection reveals that the differences are 
more apparent than substantive. 

In general terms. Performance Based Methods define what is to be accomplished rather 
than how the objective will be achieved. Some proposed definitions are: 

1) a method that produces data that meets specifications (and does not rely upon a 
prescription to achieve this goal); 

2) a monitoring approach that permits the use of appropriate analytical methods 
that meet established, demonstrated method performance standards, based on 
agency (customer) data quality (objectives) needs. Minimum required elements of 
perfonnance such as precision, accuracy, sensitivity, apecificity and detection 
(criterion) limits must be specified and an available method must be shown to meet 
the method performance standards; 

3) An analytical procedure containing the necessary method performance 
specifications to assure the desired quality of performance; 

4) An analytical method containing the necessary method performance 
specifications to assure the desired quality of performance. 



One function of any definition is to enable the differentiation of the defmed entity from all 
other entities. None of these definitions is particularly good at separating prescription 
methods from PBM's. Many proponents of prescription methods would proclaim that their 
methods contain the necessary performance specifications and thus fall within the 
definition. 
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What is the purpose for the definition? To provide more flexibility in the choice of analytical 
methods. In order to do this we must have a system that provides some assurance that all 
permissible choices meet agreed upon standards and that these standards exist for all 
relevant criteria. 

lliere are two main instances where an opportunity to be flexible is encountered, changing 
an old method and selecting a new one. Prescription methods, of course, do not allow 
either. The PBM system must provide criteria and standards that can assist in the 
determination of whether a change has been made to an old method or whether a new 
method has been adopted or created. The system must also assist in the selection of new 
methods. 

Another opportunity for flexibility can be created by separating the sample preparation 
stage from t^e analyte determination stage. PBM systems must be able to deal with this 
modularization of the analytical system. Other subsystems which are oflen modularized 
are the sampling, aliquoting and data reporting operations. 

Following this path of reasoning leads us to a provisional definition of a Performance 
Based Method: 

*A method that is part of a quality management system and is primarily 
based on a strict definition of the outcome of it's application and which 
contains criteria and standards for the creation of guidelines for how the 
outcome will be achieved". 

Tlie idea of a PMB as part of a larger system is critical to this definition and is one of the 
attributes of the method that differentiates it from a prescription method. As part of a 
quality management system the PMB lives in relation to other parts of the system like; 

• Data Quality Objectives which determine the 'strict definition of the outcome' of the PBM; 

• Quality of Design* and sound analytical principals which determine what the 'guidelines 
for how the outcome will be achieved'; 

• QC module; 

• QA module; 

• Data Reporting module; 

• Quality of Design Loop* ; 

• Method, Laboratory and Technologist Accreditation 

• Traceability 

• etc. 

This brings us to the subject of the model or format for such a method. 



* When a method undergoes lignificant modificatioo or is first created the Q of D documentation can be 
quite lengthy and is probably best kept as a separate but permanently linked file. 

* This loop fonns the basis for Continuous Quality ImprovemcnL The QC data is rolled up into QA 
documentadoa as the method is applied. Periodically, the issues highlighted by the QA system are 
addressed by a method review and update. This may, for example, result in the ideodfication of new 
sources of determinate error and thus new QC procedures. These constitute a method change and must be 
documented in the Q of D file. 
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What Do Performance Based Methods Look Like? 

Well, based on the above they should look flexible and modular. More than this, since the 
Performance Based Method approach is really a system, of which the method is only one 
component, PBM's must have ways and means of interfacini; with the other modules in 
the system. There are a number of proposals for the format of performance based methods. 
None seem to have received g;eneral approval. The following model introduces a suggestion 
for a PMB format 

An outline of what a PBM may look like is: 

INTRODUCTION 

The introduction could describe the reason for the method, the expected use of the data and 
the limitations of the method. 



General Sample Information: 

"Hiere are usually three groups that are interested in sample information 1) the sampler, 2} 
the analyst and 3) the data user. In some situations it may be helpful to further define a) 
field information that is important for data interpretation and b) information that is of 
primary importance to laboratory staff i.e. information related to safety or the presence of 
possible sources of determinate error. This level of categorization should help with 
administration and management of the PBM. 

The following table lists a some examples of these information elements (information flows 
from left to right): 



Sampler 




Analyst 


Data User 




Field 


Lab 


Field 


Ub 


Field 


Ub 


safety 


homogeneify 


aliquot 


safety 


geographical 


data qualifiers 


shipping 


phase info 


etc. 


method attributes 


regulatory 


database 


container 


preservatives 




QC 


admin codes 


report format 


custody 


date sampled 




QA 


comments 


dates 


sample size 


replicates 




analyst info 


etc. 


comments 


protocol 


notes 




storage&retrieval 




etc. 


etc 


etc. 




comments 
etc. 







These are just a few of the information items that may be necessary to specify in a PBM. 
By adopting this, or some other, categorization scheme the PBM can be more easily 
managed. For example, if the PBM is to be employed in a compliance monitoring situation 
where a continuous feedback process control model is adopted there will be little or no 
sampler functions. The PBM is simply formatted without the checklist items under 
'Sampler*. 

Note that these items constitute the criteria that a PBM would require but that each of 
these must be defined by associating them with standards, procedures and documentation / 
audit protocols. These criteria and standards form part of the guidelines for how the 
outcome mil be achieoed. 
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Range of Data Quality Objectives Serviced 

Data Quality Objectives are associated vvith a project or with a particular clients sample(s). 
It is these objectives that provide the PBM with a 'strict definition of the outcome of 
it's application'. Since a PBM must satisfy a number of DQO's and since these are not, 
in the general case, known prior to the development or modification of the method it will be 
necessary to specify within the PBM a range of DQO's for which the method is expected or 
has demonstrated capabilities. Moreover, the PBM should indicate the cost/benefit of each 
of the possible choices which could be selected when adapting the PBM to achieve the 
required DQO's. Thus, because the purpose of the PBM system is to provide flexibility, an 
important component of the method is a eoat/benefU ttn€dy»u. "Hiis analysis serves as a 
basis for selecting a specific arrangement of the available alternatives. 



Technology Specification 

This section should include two analyses, an analysis of error and a cost/benefit analysis. 

Allowable technologies e.g. ICP/OES or ICP/MS should be listed along with the sources of 
determinate error for each of the technologies. Generic QC requirements should be 
associated with each potential source of error e.g.; 

• ICP/OES; Spectral-Overlap, InterElement Correction Factors Must Be Employed 
and residual indeterminate error must be propagated to final result 

• ICP/MS; Molecxilar Interferences, species must be unambiguously monitored and 
corrections applied and residual indeterminate error must be propagated to final 
result 

• All Technologies; reagent blank contamination is a potential source of error, carry 
at least three reagent blanks and apply QC protocol as specified in SOP RB95-DE' 

Where applicable, a cost/benefit analysis should be included so that people have a basis for 
choosing the technology. In analytical chemistry this usually involves a tradeoff between 
cost on the one hand and speed and accuracy on the other. 

END OP INTRODUCTION 

A separate section may be necessary for each parameter. Each section could consist of; 

Sectionl; 
Parameter 

The exact form of the parameter should be defined along with the reporting format Issues 
like stability, competing chemistries and interpretation issues should be dealt with. 

Calibration Information 

a) Range of Valid Calibration 

b) Detection Capabilities 

c) Low Level Reporting 



' As with prescription based methods Standard Operating Procedures are a valuable tool for systematizing 
and documenting procedures that extend across many methods. 
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Calibration of the Method 

Calibration is the process of establishing the stimulus-response relationship of a well 
defined optimized analytical system. This is the point in the life of the method when 
external validation begins. The actual process for achieving this must be documented and 
subsequent standardization should be evaluated by reference to the calibration. 
Calibration is usually more exhaustive than standardization because it is a means of 
establishing both the external validity of the system and the dynamic range of the 
calibration curve. 

Standardization of the Method (Relative Methods) 

Standardization is the process of bringing an analytical system back into compliance with 
the calibration. It is usually less exhaustive than calibration. Some methods are quite 
accurate using multipoint calibration and only one or two point standardization. Others are 
calibrated with a series of matrix matched standards but are standardized with a 
completely different matrix. This is quite valid if it can be proven that there are no sources 
of error associated with the matrix and the function of the standardization is merely to re- 
establish the stimulus response relationship with a known and constant stimulus. 

In any case a means to relate daily standardization to the initial calibration is necessary 
and this must be documented. In the many cases, particularly with prescribed methods, the 
stimulus-response relationship is established using other than calibration quality 
standards and the method is externally validated by simply exposing it to performance 
evaluation by sample produced by an external agency. This will be necessary but in many 
cases not sufficient for PBM's because external validation will assume increased 
importance as multiple technologies, each with their own sources of determinate error, will 
be available as choices for the method. 



Verification of Performance 

This section is similar to what is found in most prescription based methods i.e. statements 
of expected (based on propagation of error calculations) or demonstrated (based on observed 
performance) measures of the capability of the method to estimate central tendencies and 
disparsions (accuracy and precision). Analysis of reference materials, performance 
evaluation samples, surrogate recovery, spikes, and duplicates all have a role to play here. 

These figures are attributes of the method as are detection limits. It is important to deal 
with detection limits as performance indicators and to think of them as method attributes. 
They should thus be considered in this section and separate from the section dealing with 
data reporting and data qualifiers. In the general case they are not attributes of the 
sample. In the specific case, when it can be proven that all samples are drawn from the 
same population as those employed to establish the limits, they could be used as data 
qualifiers but the issue of low level reporting is clouded by the reality of detection Vs 
quantification. 

Specificity and selectivity are also attributes of the method and should be discussed here. 



Quality Control Requirements 

Quality control requirements should be associated with known sources of error so that the 
most efficient action can be taken in the event of a failure. The PBM should specify the 
procedure and clearly identify it as such. The standard(s) for evaluation of the outcome of 
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the QC teat should be clearly stated as should the action(s) to be taken along with the 
necessary documentation protocols. The procedure should specify which information is to 
be passed to the QA system and how this is to be accomplished. Laboratory managers may 
wish to assign responsibilities for these tasks within the PBM. 



Quality Assurance Requirements 

The data which is passed from the QC system is dealt with here. In addition, things like 
accreditation, certification of laboratory & analyst skills and proficiency requirements 
should be summarized. 

References 

Appendix 

Perhaps an example method? 

A helpful addition may be a flowchart illustrating the various options for assembling the 

PBM. 



What Are Some of the Benefits of PBlVrs? 

FLEXIBLE ALTERNATIVE: To answer this question it is necessary to know what is 
wrong with the current approach and this in turn requires one to have a pretty good 
appreciation for what the 'current approcu:h' is. I suppose that there are many 'current 
approaches' but for the purposes of this discussion the current approach that serves best as 
a reference point for PBMs could be labeled the 'recipe' approach. In this model the 
overriding assiunption is If a method is sufficiently well developed and clearly written then 
someone with adequate training and experience will obtain accurate results if they follow 
the method exactly. The difficulty is; 1) this sometimes produces results that are not fit for 
the use intended and, 2) even if it does work the method becomes entrenched and there is 
little flexibility when new technology is developed. 

Ilie introduction of new technology under the 'recipe' approach is difficult New approaches 
are generally held for review until sufficient experience was accumulated for a decision to 
be made. ICP-MS, SFE, and microwave sample decomposition and extraction are some of 
the technologies which were impeded by this approach. 



ASSIGNMENT OF RESPONSIBILnY FOR QUALnY: Commercial laboratories are 
designed along the manufacturing model. They produce a product called a result. This 
commodity must be produced in the shortest possible time for the lowest possible cost and 
they must receive Uie highest possible price for it. The best way to do this is with a 
manufacturing process that is well defined, i.e. a prescribed method. Moreover, with the 
prescriptive method approach the purchaser of this commodity has an interest in paying 
the laboratory to follow the procedure rather than get an accurate result. This creates less 
problems for the consimier particularly when the produced result is biased low because it 
allows an effective defense; "we followed the procedure'. 

One of the most serious shortcomings of the 'recipe' approach is that some laboratories are 
producing data that is biased or otherwise compromised. This can happen even when an 
approved method is followed exactly and the QC is within acceptable limits. This is 
because the samples may carry a source of determinate error that was not anticipated at 
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method development time. The laboratory personnel may suspect such bias but with the 
client can place considerable fmancial pressure on the laboratory to report the result as 
less than some regulated value EVEN WHEN IT IS SUSPECTED THAT THE SAMPLE 
DOES NOT COMPLY. The client has some justification in doing this, rationalizing that 
they followed the method to the letter and it is not their fault if the EPA or the method 
development agency was too stupid to anticipate all possible sources of determinate error 
in their samples'. I have been told by lab managers that they have been instructed by their 
client not to carry out, or at least not to document, any further investigations involving 
their samples. I suspect that this practice is more wide spread than is generally 
acknowledged. This verges on illegal, is certainly not in the spirit of the law and is, by most 
standards, unethical! 

The PBM approach places the emphasis for ensuring quality (a.k.a. fatness for purpose) 
with the people producing the result (samplers, handlers, and analysts) rather than 
primarily focusing on the method as the vehicle for delivering quality data. The recipe 
approach tends to push analysts into the role of measurement technologists whereas the 
PBM approach tends to broaden the responsibilities of the analyst and rely more on the 
skills of analytical chemistry. T^is is at once the strength and the weakness of the PBM 
approach. By empowering the analyst to make decisions about quality during the analytical 
process we place the decision making and the responsibility in a more effective strategic 
position. For me, one of the central issues of is this one: do we rely primarily upon the 
people developing and managing the method, or do we rely upon the analytical 
technologist? With the PBM approach we can do both. 

The benefits of this are: 

1) clients can use a wider variety of laboratories thus providing competition and 
flezibihty; 

2) lab managers have more latitude in choice of resources and their management; 

3) techiKtlogy developers are less constrained when allocating resources for 
resaarch; 

4) analytical chcxnista are less constrained when developing or modifying 
methods; 

5) the increased emphasis on the analyst for the provision of quality data should 
produce more flexible, qualified and empowered analysts which in turn should 
produce hij^er quality data, particularly in cases where the sample carries a source 
of error that was not adequately controlled in the method. 

Summary: 

• quality is defined in a more effective way; 

• analyst and management flexibility; 

• encourages method and technique development; 

• more efRcient and better methods A results; 

• more highly akilled amalysts. 



What Are Some of the Liabilities of PBM's? 

COST: Many people disagree with the PBM approach. It is certainly more difficult to 
participate in a S3rstem where you are responsible for the outcome and are not simply 
checked with respect to 'following the steps'. Methods must be developed, chosen and 
deployed so that a quality result is produced. This means that laboratory managers and 
analysts must demonstrate competencies that extend well beyond simple implementation 
and cariying out of prescribed procedures. Analyte, level of concentration, matrix and any 
other factors that could carry determinate error become of primary concern to these people 
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because the responsibility is no longer essentially with the agency prescribing the methods. 
The development of these competencies will cost both time and money. Moreover, the power 
for implementation of a quality system that involves PBM's resides with lab management. 
It will cost time and money to develop the expertise and the systems. The solution to this 
concern may emerge from a realization that not all situations warrant a PBM solution. 
Prescription based methods will continue to be the most cost effective and probably the 
most accurate when applied to samples of well characterized and homogenous composition 
and when the technologies applied are mature and well accepted. PBM's will be necessary 
and may even be more cost effective when samples are less well characterized or when new 
technologies are being applied to the problem. Soma examples of where one approach may 
be more suitable over the other are: 

R«.mpl*> Approach 

1) homogenous process-stream monitoring prescription 

2) high impact, well defined, complex testing like AIDS etc. prescription 

3) long term programs where trending is more important than prescription 
absolute accuracy 

4) situations where the result is defmed by the method e.g. 'ether prescription 
eztractable oils' 

6) non-homogenous, difficult to characterize sample streams PBM 

6) poorly defined testing, e.g. qualitative screening of dump sites PBM 

7) short term projects where absolute accuracy is required across PBM 
laboratories, sampling sites, jurisdictions, methods etc 



CULTUKE: When managers implement a quality system that is built around prescribed 
methods they generally are rewarded by becoming more "plugged in' to the information. It 
ii easy to believe that the method is a 'constant' the attributes of which can be known. 
Some will interpret the concept of PBM as being a loosening of standards and re-label their 
prescriptive methods 'PBM*. "niis could result in a lowering of the quality of the results 
from these methods. 

COMPLEXITY: Accreditation may become the savior for some. This is a dubious asset 
because, by itself, accreditation does not guarantee quality. Indeed, accreditation is largely 
concerned with quality of conformance not quality of design. Accreditation ensures that the 
procedure has been followed as prescribed but it does tittle to ensure quality of design. 
Experience has shown that accreditation can be a wall to hide behind. 

ABSENCE OF GUIDELINES, CRITERIA AND STANDARDS: It will be important to 
devise an agreed upon protocol whereby a new PBM can be demonstrated to be valid and 
throu^ which it can be characterized i.e. what are the attributes of a PBM? Test method 
equivalency is probably a necessary but insufficient criteria. When criteria for establishing 
validity have b«en agreed upon it will be necessary to formulate standards under each of 
the criteria or at least to have a guideline for developing such standards. 

Summary: 

• increased coat to develop; 

• change in culture (usually painful for someone); 

• increased complexity of implementation and administration; 

• guidelines, criteria and standards must be developed and agreed upon. 

What Are Some of the Issues And Approaches? 
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Some relevant questions are: 

1) who are the stakeholders? 

2) what are the guidehnes? 

2a) which criteria? 
2b) what standards? 

3) who develops them? 

4) who assures that they are adhered to? 

STAKEHOIDEBS: Among the stakeholders are data users (e.g. regulators, lawyers, 
engineers, biologists), laboratory managera, analytical chemists and technicians, 
instrument companies and suppliers. There is a wide variety of needs within each of these 
categories and the guidelines must provide a consistent framework for everyone to take 
effective action. 



GUIDELXNES: Guidelines will be required for the various aspects of the PBM system: 
preparation of DQO's, Specification of Method Performance Criteria, Specification of 
Method Audit and Proficiency, Specification of Reporting Protocols. In order to achieve th* 
goal of flexibility these guidelines should focus on trapping and correcting error as well as 
ensuring that the correct information (not just data) is generated and communicated 
appropriately. 



CRTTEBIA * STANDARDS: Some Suggestioiu For Analytical Methods 

To provide the benedta that we have discussed and to avoid the liabilities PBM systems 
must have guidelines with criteria and standards that enable their effective use. These 
should assist in deciding if the PBM approach is best and should also provide a way to 
ensure the quality of the method a) when developed (Quality of Design) and b) when being 
dosen and c) when being applied (Quality of Conformance). 

WE BCUST BE CAREFUL WHAT WE STANDARDIZE: Some groups are proceeding in 

the ri^t direction but they are still searching for a prescription to achieve their goals 

rather than guidelines that can be used to define a system. For example, in a doctmient 

asking for pi^lic participation the discussion topics were: 

'establishment of standardized QC for analytical methods'; 

and 

'establishment of standardized data elements for reporting analytical results'. 

The document proceeded to list a set of mandatory QC procedures like spikes, duplicates 
reagent blanks and others that would be necessary to include in a PBM. This begins to look 
less like guidelines and more like a prescription. Instead it may be more effective to ask: 
"What possible sources of determinate error are associated with this PBM'? If reagent 
blank contamination can never aiTect results then the inclusion of reagent blanks is a 
waste of money and time. The focus should be on tiie sources of error, their control and 
assurance by setting guidelines for handling error and specifying criteria and standards. 

'Standardized QC tests' to demonstrate that a method change, adoption or development has 
not degraded method performance may not be the best approach because QC, by definition, 
is directed at problems of conformance not at problems of design. During the process of 
method change, adoption, or development the focus should be on Quality of Design and the 
PBM approach should provide guidelines for assurance of the design process. Once the 
PBM is in place quality of conformance becomes an issue but from the point of view of the 



practitioner the method looks very much like a prescription method. Once QC procedures 
have been linked to known sources of error (and therefor are associated with action 
procedures) the 4-step QC process i.e. 1) test, 2) inspect/compare, 3) take action, and 4) 
document work in much the same manner as with prescription methods. With PBM's it will 
be even more important than with prescription based methods to periodically feed the QC 
information into the Quality of Design loop. 



METHOD ATTRIBUTES V« DATA QUALIFIERS: The PBM itself should carry those 
data reporting specifications imposed by the method attributes e.g. soluble / insoluble, 
MDL, typical uncertainties*, etc.) and it would carry only those QC procedures that are 
technique specific (e.g. lECs in ICP/OES). Data Qualifiers should not be proclaimed in the 
method but should be determined by the DQO's for the clients who receive data from the 
method. 'Rie PBM should be able to feed data into a reporting subsystem that does not 
change the data with respect to those attributes imposed by the method but should append 
data qualifiers that are required by each client For example, a PBM may produce Cd 
concentrations in ground water. The core method should determine the MDL (a method 
attribut«) but the low level reporting requirements may be difTerent for different clients. 
The data is still the same and the MDL is independent of client However the data 
qualifiers may be difTerent for different clients and one may want to see a result like 0.034 
<MDL(.05) <Provincial Water Quality Criteria (.2) whereas another client may wish to see 
a result like 0.034 <RDL(.5). "Hie separation of method attributes from data qualifiers 
allows for clearer criteria and standards for PBM'a. 



DIFFERENTIATE THE METHOD: For similar reasons it may be wise to separate the 
sampling, aub-sampling and analysis phases into separate methods. This should make the 
task of tracing wher« errors were introduced when a change has resulted in compromised 
data. Also, the objective of flexibility is achieved by allowing managers and analysts to 
choose from various preparation methods and stream the samples to a particular 
measurement method. Measurement methods based on difTerent technologies are in 
general different methods because the sources of both determinate error and indeterminata 
error differ. Using similar criteria, preparation methods that carry fundamentally different 
sources of error require different QC protocols. Therefor they are different If the analyst 
can argue and demonstrate that the sources of error for the two techniques, when applied 
to a specified set of samples, are the same then the difference is a method change not a new 
method. 



CALIBRATION A STANDARDIZATION: Another opportunity for improving PBM's is to 
clearly separate the calibration process from the standardization process. This would 
provide assurance that difTerent PBM's are at least externally validated and may result in 
more effective use of international certified reference materials. 



* an excellent treatment of uncertainties and how they can be used is provided in the document A Guide 
To The Expression of Uocercainly In Measurement. ISO Geneva 1993. 
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LINK QUALITY CONTROL TO SOURCES OF DETERMINATE ERROR: PBM's ara 
supposed to allow flexibility while ensuring quality and this quality is, to a large extent, 
the responsibility of the laboratory stafT. The prescription based approach attempts to 
measure quality into a product by specifying a more or less comprehensive list of quality 
control procedures which had to be followed. To provide flexibility and cost effectiveness in 
the PBM approach QC should be linked to sources of determinate error. For example, if 
experiments performed at method development time have demonstrated that proportional 
determinate error is the main source of error than QC procedures like taking different 
aliquot sizes or performing spikes is not only a waste of time and money it is misleading 
because the analyst would interpret the results as being accurate when in fact some action 
should have been taken. 

PBM's should also be more rigorous with respect to how QC data is dealt with. The four 
basic steps of QC should be an explicit part of the method i.e. 1) test/measure, 2) compare 
against expected (usually a Statistical Process Control population parameter), 3) take 
action (or not), and 4) document "Hie action will be considerably more effective if the QC 
procedure is linked to sources of determinate error. 



LINK QUALITY ASSURANCE TO A QUALITY OP DESIGN LOOP: The Quality 
Assurance system (among other things) is the repository of the results of the QC 
subsystem. The documentation listing or mmmarizing the QC performance of a PBM 
should be periodically reviewed and an analytical audit of the method should be performed 
with a view to making it more efficient and to eliminating or reducing the sources of error 
that have been identified during the last period. This helps to create a culture of continuous 
quality improvement which I believe will be crucial to the success of the PBM approach. 

DQO'a BECOME VERY IMPORTANT: Since PBM's plug into the scheme after the 
DQO's the PBM approach will need guidelines for the specification of DQO's so that the 
PBM can be linked with the users needs. If this link is not clear the person(s) responsible 
for choosing, adopting or developing the method will have Uttle basis for their activities. 



DEVELOPMENT OF PERFORMANCE CRITERIA AND INDICATORS: DQO's an a 
tools for the user to specify the quality of results that are required. Performance criteria 
are tools for the analytical chemist and laboratory manager to communicate the capability 
of the process (method) that they are offering. DQO's should live outside the method, PC 
should live inside the method, l^ey should link in a manner that enables data users and 
laboratory staff to agree upon the method that will be used to achieve the objectives. Tlie 
speciHc Performance Indicators used that would be selected for each of the criteria would 
depend on the DQO's of the client and the PC's of the method. Some Performance 
Indicators are: detection, accuracy, precision, specificity, selectivity, and ruggedness with 
respect to various parameters. 

PROFICIENCY MONITORING: Since QC procedures are flexible and are linked to 
sources of error that are not only technique and method specific but may also be laboratoiy 
and personnel specific it wall be necessary to validate PBM performance with a proficiency 
monitoring system. This includes things like method performance evaluation using test 
samples, skills and competency monitoring for analysts as well as broad method audits to 
ensure that guidelines and procedures are followed and that data reporting is correct 
(particularly QA of software used for this purpose). 
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Tha above suggestions are by no means a comprehensive list of PBM criteria but they are 
things to consider when developing guidelines for a PBM based system. Who will develop 
these guidelines and who will ensure that they are being followed? Well, the quick answer 
is 'I don't know*. I suspect that it will continue to be a bottom-up process until the concepts 
are clear enou^ for extreme positions to be taken. At this point a group or agency may be 
able to synthesize a PBM system that is generally acceptable or adaptable to the analytical 
community. Until this happens specific data users and laboratory managers will continue 
to experiment with the concepts. 

Who Mi?ht Use PBlVrs? 

"Hie EPA, Office of Solid Waste has been using PBM's for more than 10 yean and they are 
working to promote the philosophy throU^out the agency. EIPA Office of Science and 
Technology (within EPA O^fEice of Water) is reviewing ways to increase flexibility of method 
change in existing '40 CFR Part 136 test methods'. The MOEE MISA program took this 
approach from the beginning of the program. The USEPA Contract Ltdx>ratory Program 
(CLP) employs the PBM approach when the required work does not have a routine method 
that could be prescribed. 

PBM systems are probably not the best approach in all cases. Prescription methods will 
continue to be relied upon for analysis when the problem is well defined and mature, coet 
effective methods exist. "Rie PBM approach will continue to flourish when stakeholders are 
working with less well defined problems, when new technology is being incorporated, when 
clear quality of conformance protocols cannot be easily applied and when quahty is more 
dependent upon the experience and judgment of the practitioner than upon strict 
adherence to a procedure. 



Where Do W e Go From Here? 

There is a qualitative difference between a Performance Baaed Method and a prescription 
method. Both work best when viewed as part of a larger overall system but PBM's are 
critically dependent on this system whereas prescription methods are more independent of 
the Quality Management system. 

The goal is to provide a framework for the development and use of methods that produce 
quaUty results in a reasonable time and at the lowest possible cost. The issue is really not 
which approach is better but which is better in a particular case. I suspect that the so- 
called 'prescription approach' will continue to be used where it is appropriate and that the 
PBM approach will continue to grow and be successful in cases where it is most effective, 
llie debate should continue, not on which approach is better but on defining the PBM 
approach and developing more powerful conceptual tools to augment it's development The 
experience of well implimented PBM systems will determine how rapidly and extensively 
the approach gains acceptance. 
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